{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A-1，Kaggle免费GPU使用攻略"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 一，注册Kaggle"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在国内使用邮箱注册kaggle时会遇到一个人机验证的步骤，可以通过翻墙访问外网的方式完成，但比较麻烦。\n",
    "\n",
    "推荐使用FireFox浏览器，下载Header Editor进行解决，无需翻墙相对简单。\n",
    "\n",
    "1，下载安装FireFox浏览器\n",
    "\n",
    "2，添加Header Editor浏览器插件【找到FireFox浏览器的 工具 -> 扩展和主题 —> 搜索 Header Editor -> 添加到FireFox】\n",
    "\n",
    "3，配置Header Editor插件【找到FireFox右上角Header Editor -> 导出和导入 -> 下载规则中输入如下规则url -> 点击向下箭头加载  】\n",
    "\n",
    "规则url: https://azurezeng.github.io/static/HE-GoogleRedirect.json\n",
    "\n",
    "4，在kaggle官网用邮箱正常注册kaggle即可。\n",
    "\n",
    "kaggle官网：https://www.kaggle.com/\n",
    "\n",
    "5，此后就可以在任何能联网的地方正常登录kaggle，不再需要Header Editor了\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 二，设置GPU"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1，新建notebook. 【点击kaggle主页面左上角+, 选择notebook】\n",
    "\n",
    "2，开启GPU开关。【点击展开notebook右上角 |< 设置，设置Accelerator为GPU 】\n",
    "\n",
    "3，查看GPU信息。【NoteBook中使用 nvidia-smi查看】\n",
    "\n",
    "![](https://tva1.sinaimg.cn/large/e6c9d24egy1h60a2hwnf2j20hs082jtk.jpg)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-09-11T02:58:24.076598Z",
     "iopub.status.busy": "2022-09-11T02:58:24.075960Z",
     "iopub.status.idle": "2022-09-11T02:58:25.134868Z",
     "shell.execute_reply": "2022-09-11T02:58:25.133687Z",
     "shell.execute_reply.started": "2022-09-11T02:58:24.076559Z"
    }
   },
   "outputs": [],
   "source": [
    "!nvidia-smi "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-09-11T02:58:37.328627Z",
     "iopub.status.busy": "2022-09-11T02:58:37.328044Z",
     "iopub.status.idle": "2022-09-11T02:58:37.342161Z",
     "shell.execute_reply": "2022-09-11T02:58:37.340202Z",
     "shell.execute_reply.started": "2022-09-11T02:58:37.328573Z"
    }
   },
   "outputs": [],
   "source": [
    "import torch \n",
    "torch.cuda.is_available() "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 三，上传数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1，点击展开notebook右上角 |< 设置，找到 Add Data，可以从Kaggle社区发布的数据集中选择一些想要的数据集。\n",
    "\n",
    "2，也可以选在代表上传的向上箭头，上传数据集文件作为自定义数据集。建议压缩后上传，传输效率较高。\n",
    "\n",
    "3，此外，也可以通过把数据放在github项目中，用git clone的方式获取。\n",
    "\n",
    "4，notebook加载进来数据集后，可以在右边数据文件位置点击获取对应路径。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](https://tva1.sinaimg.cn/large/e6c9d24egy1h60albtrxij20as0aimxc.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br>\n",
    "\n",
    "<font color=\"red\">\n",
    " \n",
    "公众号 **算法美食屋** 回复关键词：**pytorch**， 获取本范例所用数据集eat_pytorch_datasets百度云盘下载链接。\n",
    "    \n",
    "</font> \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 四，CPU训练代码\n",
    "\n",
    "约14s一个Epoch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-09-11T03:00:57.967345Z",
     "iopub.status.busy": "2022-09-11T03:00:57.966948Z",
     "iopub.status.idle": "2022-09-11T03:02:09.779414Z",
     "shell.execute_reply": "2022-09-11T03:02:09.778142Z",
     "shell.execute_reply.started": "2022-09-11T03:00:57.967311Z"
    }
   },
   "outputs": [],
   "source": [
    "import torch \n",
    "from torch import nn\n",
    "from torch.utils.data import Dataset,DataLoader\n",
    "from torchvision import transforms as T\n",
    "from torchvision import datasets \n",
    "\n",
    "\n",
    "#======================================================================\n",
    "# 一，准备数据\n",
    "#======================================================================\n",
    "\n",
    "transform_img = T.Compose(\n",
    "    [T.ToTensor()])\n",
    "\n",
    "def transform_label(x):\n",
    "    return torch.tensor(x)\n",
    "\n",
    "ds_train = datasets.ImageFolder(\"../input/eat-pytorch-datasets/eat_pytorch_datasets/cifar2/train/\",\n",
    "            transform = transform_img,target_transform = transform_label)\n",
    "ds_val = datasets.ImageFolder(\"../input/eat-pytorch-datasets/eat_pytorch_datasets/cifar2/test/\",\n",
    "            transform = transform_img,target_transform = transform_label)\n",
    "print(ds_train.class_to_idx)\n",
    "\n",
    "dl_train = DataLoader(ds_train,batch_size = 50,shuffle = True, pin_memory=True, num_workers = 8)\n",
    "dl_val = DataLoader(ds_val,batch_size = 50,shuffle = False, pin_memory=True, num_workers = 8)\n",
    "\n",
    "\n",
    "%matplotlib inline\n",
    "%config InlineBackend.figure_format = 'svg'\n",
    "\n",
    "#查看部分样本\n",
    "from matplotlib import pyplot as plt \n",
    "\n",
    "plt.figure(figsize=(8,8)) \n",
    "for i in range(9):\n",
    "    img,label = ds_train[i]\n",
    "    img = img.permute(1,2,0) \n",
    "    ax=plt.subplot(3,3,i+1)\n",
    "    ax.imshow(img.numpy())\n",
    "    ax.set_title(\"label = %d\"%label.item())\n",
    "    ax.set_xticks([])\n",
    "    ax.set_yticks([]) \n",
    "plt.show()\n",
    "\n",
    "# Pytorch的图片默认顺序是 Batch,Channel,Width,Height\n",
    "for features,labels in dl_train:\n",
    "    print(features.shape,labels.shape) \n",
    "    break\n",
    "    \n",
    "    \n",
    "    \n",
    "\n",
    "#======================================================================\n",
    "# 二，定义模型\n",
    "#======================================================================\n",
    "\n",
    "class Net(nn.Module):\n",
    "    \n",
    "    def __init__(self):\n",
    "        super(Net, self).__init__()\n",
    "        self.conv1 = nn.Conv2d(in_channels=3,out_channels=32,kernel_size = 3)\n",
    "        self.pool = nn.MaxPool2d(kernel_size = 2,stride = 2)\n",
    "        self.conv2 = nn.Conv2d(in_channels=32,out_channels=64,kernel_size = 5)\n",
    "        self.dropout = nn.Dropout2d(p = 0.1)\n",
    "        self.adaptive_pool = nn.AdaptiveMaxPool2d((1,1))\n",
    "        self.flatten = nn.Flatten()\n",
    "        self.linear1 = nn.Linear(64,32)\n",
    "        self.relu = nn.ReLU()\n",
    "        self.linear2 = nn.Linear(32,2)\n",
    "        \n",
    "    def forward(self,x):\n",
    "        x = self.conv1(x)\n",
    "        x = self.pool(x)\n",
    "        x = self.conv2(x)\n",
    "        x = self.pool(x)\n",
    "        x = self.dropout(x)\n",
    "        x = self.adaptive_pool(x)\n",
    "        x = self.flatten(x)\n",
    "        x = self.linear1(x)\n",
    "        x = self.relu(x)\n",
    "        x = self.linear2(x)\n",
    "        return x \n",
    "        \n",
    "net = Net()\n",
    "print(net)\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "#======================================================================\n",
    "# 三，训练模型(CPU)\n",
    "#======================================================================\n",
    "\n",
    "\n",
    "import os,sys,time\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import datetime \n",
    "from tqdm import tqdm \n",
    "\n",
    "import torch\n",
    "from torch import nn \n",
    "from copy import deepcopy\n",
    "from torchmetrics import Accuracy\n",
    "#注：多分类使用torchmetrics中的评估指标，二分类使用torchkeras.metrics中的评估指标\n",
    "\n",
    "def printlog(info):\n",
    "    nowtime = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')\n",
    "    print(\"\\n\"+\"==========\"*8 + \"%s\"%nowtime)\n",
    "    print(str(info)+\"\\n\")\n",
    "    \n",
    "\n",
    "net = Net()\n",
    "\n",
    "loss_fn = nn.CrossEntropyLoss()\n",
    "optimizer= torch.optim.Adam(net.parameters(),lr = 0.01)   \n",
    "metrics_dict = {\"acc\":Accuracy()}\n",
    "\n",
    "epochs = 5 \n",
    "ckpt_path='checkpoint.pt'\n",
    "\n",
    "#early_stopping相关设置\n",
    "monitor=\"val_acc\"\n",
    "patience=3\n",
    "mode=\"max\"\n",
    "\n",
    "history = {}\n",
    "\n",
    "for epoch in range(1, epochs+1):\n",
    "    printlog(\"Epoch {0} / {1}\".format(epoch, epochs))\n",
    "\n",
    "    # 1，train -------------------------------------------------  \n",
    "    net.train()\n",
    "    \n",
    "    total_loss,step = 0,0\n",
    "    \n",
    "    loop = tqdm(enumerate(dl_train), total =len(dl_train))\n",
    "    train_metrics_dict = deepcopy(metrics_dict) \n",
    "    \n",
    "    for i, batch in loop: \n",
    "        \n",
    "        features,labels = batch\n",
    "        #forward\n",
    "        preds = net(features)\n",
    "        loss = loss_fn(preds,labels)\n",
    "        \n",
    "        #backward\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "        optimizer.zero_grad()\n",
    "            \n",
    "        #metrics\n",
    "        step_metrics = {\"train_\"+name:metric_fn(preds, labels).item() \n",
    "                        for name,metric_fn in train_metrics_dict.items()}\n",
    "        \n",
    "        step_log = dict({\"train_loss\":loss.item()},**step_metrics)\n",
    "\n",
    "        total_loss += loss.item()\n",
    "        \n",
    "        step+=1\n",
    "        if i!=len(dl_train)-1:\n",
    "            loop.set_postfix(**step_log)\n",
    "        else:\n",
    "            epoch_loss = total_loss/step\n",
    "            epoch_metrics = {\"train_\"+name:metric_fn.compute().item() \n",
    "                             for name,metric_fn in train_metrics_dict.items()}\n",
    "            epoch_log = dict({\"train_loss\":epoch_loss},**epoch_metrics)\n",
    "            loop.set_postfix(**epoch_log)\n",
    "\n",
    "            for name,metric_fn in train_metrics_dict.items():\n",
    "                metric_fn.reset()\n",
    "                \n",
    "    for name, metric in epoch_log.items():\n",
    "        history[name] = history.get(name, []) + [metric]\n",
    "        \n",
    "\n",
    "    # 2，validate -------------------------------------------------\n",
    "    net.eval()\n",
    "    \n",
    "    total_loss,step = 0,0\n",
    "    loop = tqdm(enumerate(dl_val), total =len(dl_val))\n",
    "    \n",
    "    val_metrics_dict = deepcopy(metrics_dict) \n",
    "    \n",
    "    with torch.no_grad():\n",
    "        for i, batch in loop: \n",
    "\n",
    "            features,labels = batch\n",
    "            \n",
    "            #forward\n",
    "            preds = net(features)\n",
    "            loss = loss_fn(preds,labels)\n",
    "\n",
    "            #metrics\n",
    "            step_metrics = {\"val_\"+name:metric_fn(preds, labels).item() \n",
    "                            for name,metric_fn in val_metrics_dict.items()}\n",
    "\n",
    "            step_log = dict({\"val_loss\":loss.item()},**step_metrics)\n",
    "\n",
    "            total_loss += loss.item()\n",
    "            step+=1\n",
    "            if i!=len(dl_val)-1:\n",
    "                loop.set_postfix(**step_log)\n",
    "            else:\n",
    "                epoch_loss = (total_loss/step)\n",
    "                epoch_metrics = {\"val_\"+name:metric_fn.compute().item() \n",
    "                                 for name,metric_fn in val_metrics_dict.items()}\n",
    "                epoch_log = dict({\"val_loss\":epoch_loss},**epoch_metrics)\n",
    "                loop.set_postfix(**epoch_log)\n",
    "\n",
    "                for name,metric_fn in val_metrics_dict.items():\n",
    "                    metric_fn.reset()\n",
    "                    \n",
    "    epoch_log[\"epoch\"] = epoch           \n",
    "    for name, metric in epoch_log.items():\n",
    "        history[name] = history.get(name, []) + [metric]\n",
    "\n",
    "    # 3，early-stopping -------------------------------------------------\n",
    "    arr_scores = history[monitor]\n",
    "    best_score_idx = np.argmax(arr_scores) if mode==\"max\" else np.argmin(arr_scores)\n",
    "    if best_score_idx==len(arr_scores)-1:\n",
    "        torch.save(net.state_dict(),ckpt_path)\n",
    "        print(\"<<<<<< reach best {0} : {1} >>>>>>\".format(monitor,\n",
    "             arr_scores[best_score_idx]),file=sys.stderr)\n",
    "    if len(arr_scores)-best_score_idx>patience:\n",
    "        print(\"<<<<<< {} without improvement in {} epoch, early stopping >>>>>>\".format(\n",
    "            monitor,patience),file=sys.stderr)\n",
    "        break \n",
    "    net.load_state_dict(torch.load(ckpt_path,weights_only=True))\n",
    "    \n",
    "dfhistory = pd.DataFrame(history)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 五，GPU训练代码\n",
    "\n",
    "约8s一个Epoch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-09-11T03:02:14.848810Z",
     "iopub.status.busy": "2022-09-11T03:02:14.848443Z",
     "iopub.status.idle": "2022-09-11T03:02:51.924526Z",
     "shell.execute_reply": "2022-09-11T03:02:51.923309Z",
     "shell.execute_reply.started": "2022-09-11T03:02:14.848776Z"
    }
   },
   "outputs": [],
   "source": [
    "import torch \n",
    "from torch import nn\n",
    "from torch.utils.data import Dataset,DataLoader\n",
    "from torchvision import transforms as T\n",
    "from torchvision import datasets \n",
    "\n",
    "\n",
    "#======================================================================\n",
    "# 一，准备数据\n",
    "#======================================================================\n",
    "\n",
    "transform_img = T.Compose(\n",
    "    [T.ToTensor()])\n",
    "\n",
    "def transform_label(x):\n",
    "    return torch.tensor(x)\n",
    "\n",
    "ds_train = datasets.ImageFolder(\"../input/eat-pytorch-datasets/eat_pytorch_datasets/cifar2/train/\",\n",
    "            transform = transform_img,target_transform = transform_label)\n",
    "ds_val = datasets.ImageFolder(\"../input/eat-pytorch-datasets/eat_pytorch_datasets/cifar2/test/\",\n",
    "            transform = transform_img,target_transform = transform_label)\n",
    "print(ds_train.class_to_idx)\n",
    "\n",
    "dl_train = DataLoader(ds_train,batch_size = 50,shuffle = True, pin_memory=True, num_workers = 8)\n",
    "dl_val = DataLoader(ds_val,batch_size = 50,shuffle = False, pin_memory=True, num_workers = 8)\n",
    "\n",
    "\n",
    "\n",
    "%matplotlib inline\n",
    "%config InlineBackend.figure_format = 'svg'\n",
    "\n",
    "#查看部分样本\n",
    "from matplotlib import pyplot as plt \n",
    "\n",
    "plt.figure(figsize=(8,8)) \n",
    "for i in range(9):\n",
    "    img,label = ds_train[i]\n",
    "    img = img.permute(1,2,0) \n",
    "    ax=plt.subplot(3,3,i+1)\n",
    "    ax.imshow(img.numpy())\n",
    "    ax.set_title(\"label = %d\"%label.item())\n",
    "    ax.set_xticks([])\n",
    "    ax.set_yticks([]) \n",
    "plt.show()\n",
    "\n",
    "# Pytorch的图片默认顺序是 Batch,Channel,Width,Height\n",
    "for features,labels in dl_train:\n",
    "    print(features.shape,labels.shape) \n",
    "    break\n",
    "    \n",
    "    \n",
    "    \n",
    "\n",
    "#======================================================================\n",
    "# 二，定义模型\n",
    "#======================================================================\n",
    "\n",
    "class Net(nn.Module):\n",
    "    \n",
    "    def __init__(self):\n",
    "        super(Net, self).__init__()\n",
    "        self.conv1 = nn.Conv2d(in_channels=3,out_channels=32,kernel_size = 3)\n",
    "        self.pool = nn.MaxPool2d(kernel_size = 2,stride = 2)\n",
    "        self.conv2 = nn.Conv2d(in_channels=32,out_channels=64,kernel_size = 5)\n",
    "        self.dropout = nn.Dropout2d(p = 0.1)\n",
    "        self.adaptive_pool = nn.AdaptiveMaxPool2d((1,1))\n",
    "        self.flatten = nn.Flatten()\n",
    "        self.linear1 = nn.Linear(64,32)\n",
    "        self.relu = nn.ReLU()\n",
    "        self.linear2 = nn.Linear(32,2)\n",
    "        \n",
    "    def forward(self,x):\n",
    "        x = self.conv1(x)\n",
    "        x = self.pool(x)\n",
    "        x = self.conv2(x)\n",
    "        x = self.pool(x)\n",
    "        x = self.dropout(x)\n",
    "        x = self.adaptive_pool(x)\n",
    "        x = self.flatten(x)\n",
    "        x = self.linear1(x)\n",
    "        x = self.relu(x)\n",
    "        x = self.linear2(x)\n",
    "        return x \n",
    "        \n",
    "net = Net()\n",
    "print(net)\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "#======================================================================\n",
    "# 三，训练模型(CPU)\n",
    "#======================================================================\n",
    "\n",
    "\n",
    "import os,sys,time\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import datetime \n",
    "from tqdm import tqdm \n",
    "\n",
    "import torch\n",
    "from torch import nn \n",
    "from copy import deepcopy\n",
    "from torchmetrics import Accuracy\n",
    "#注：多分类使用torchmetrics中的评估指标，二分类使用torchkeras.metrics中的评估指标\n",
    "\n",
    "def printlog(info):\n",
    "    nowtime = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')\n",
    "    print(\"\\n\"+\"==========\"*8 + \"%s\"%nowtime)\n",
    "    print(str(info)+\"\\n\")\n",
    "    \n",
    "\n",
    "net = Net()\n",
    "\n",
    "loss_fn = nn.CrossEntropyLoss()\n",
    "optimizer= torch.optim.Adam(net.parameters(),lr = 0.01)   \n",
    "metrics_dict = {\"acc\":Accuracy()}\n",
    "\n",
    "\n",
    "#------------------------------移动模型到GPU上------------------------------\n",
    "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
    "net.to(device)\n",
    "loss_fn.to(device)\n",
    "for name,fn in metrics_dict.items():\n",
    "    fn.to(device)\n",
    "#-------------------------------------------------------------------------\n",
    "\n",
    "\n",
    "epochs = 5\n",
    "ckpt_path='checkpoint.pt'\n",
    "\n",
    "#early_stopping相关设置\n",
    "monitor=\"val_acc\"\n",
    "patience=3\n",
    "mode=\"max\"\n",
    "\n",
    "history = {}\n",
    "\n",
    "for epoch in range(1, epochs+1):\n",
    "    printlog(\"Epoch {0} / {1}\".format(epoch, epochs))\n",
    "\n",
    "    # 1，train -------------------------------------------------  \n",
    "    net.train()\n",
    "    \n",
    "    total_loss,step = 0,0\n",
    "    \n",
    "    loop = tqdm(enumerate(dl_train), total =len(dl_train))\n",
    "    train_metrics_dict = deepcopy(metrics_dict) \n",
    "    \n",
    "    for i, batch in loop: \n",
    "        \n",
    "        features,labels = batch\n",
    "        \n",
    "        #------------------------------移动数据到GPU上------------------------------\n",
    "        features = features.to(device)\n",
    "        labels = labels.to(device)\n",
    "        #-------------------------------------------------------------------------\n",
    "        \n",
    "        #forward\n",
    "        preds = net(features)\n",
    "        loss = loss_fn(preds,labels)\n",
    "        \n",
    "        #backward\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "        optimizer.zero_grad()\n",
    "            \n",
    "        #metrics\n",
    "        step_metrics = {\"train_\"+name:metric_fn(preds, labels).item() \n",
    "                        for name,metric_fn in train_metrics_dict.items()}\n",
    "        \n",
    "        step_log = dict({\"train_loss\":loss.item()},**step_metrics)\n",
    "\n",
    "        total_loss += loss.item()\n",
    "        \n",
    "        step+=1\n",
    "        if i!=len(dl_train)-1:\n",
    "            loop.set_postfix(**step_log)\n",
    "        else:\n",
    "            epoch_loss = total_loss/step\n",
    "            epoch_metrics = {\"train_\"+name:metric_fn.compute().item() \n",
    "                             for name,metric_fn in train_metrics_dict.items()}\n",
    "            epoch_log = dict({\"train_loss\":epoch_loss},**epoch_metrics)\n",
    "            loop.set_postfix(**epoch_log)\n",
    "\n",
    "            for name,metric_fn in train_metrics_dict.items():\n",
    "                metric_fn.reset()\n",
    "                \n",
    "    for name, metric in epoch_log.items():\n",
    "        history[name] = history.get(name, []) + [metric]\n",
    "        \n",
    "\n",
    "    # 2，validate -------------------------------------------------\n",
    "    net.eval()\n",
    "    \n",
    "    total_loss,step = 0,0\n",
    "    loop = tqdm(enumerate(dl_val), total =len(dl_val))\n",
    "    \n",
    "    val_metrics_dict = deepcopy(metrics_dict) \n",
    "    \n",
    "    with torch.no_grad():\n",
    "        for i, batch in loop: \n",
    "\n",
    "            features,labels = batch\n",
    "            \n",
    "            #------------------------------移动数据到GPU上------------------------------\n",
    "            features = features.to(device)\n",
    "            labels = labels.to(device)\n",
    "            #-------------------------------------------------------------------------\n",
    "            \n",
    "            #forward\n",
    "            preds = net(features)\n",
    "            loss = loss_fn(preds,labels)\n",
    "\n",
    "            #metrics\n",
    "            step_metrics = {\"val_\"+name:metric_fn(preds, labels).item() \n",
    "                            for name,metric_fn in val_metrics_dict.items()}\n",
    "\n",
    "            step_log = dict({\"val_loss\":loss.item()},**step_metrics)\n",
    "\n",
    "            total_loss += loss.item()\n",
    "            step+=1\n",
    "            if i!=len(dl_val)-1:\n",
    "                loop.set_postfix(**step_log)\n",
    "            else:\n",
    "                epoch_loss = (total_loss/step)\n",
    "                epoch_metrics = {\"val_\"+name:metric_fn.compute().item() \n",
    "                                 for name,metric_fn in val_metrics_dict.items()}\n",
    "                epoch_log = dict({\"val_loss\":epoch_loss},**epoch_metrics)\n",
    "                loop.set_postfix(**epoch_log)\n",
    "\n",
    "                for name,metric_fn in val_metrics_dict.items():\n",
    "                    metric_fn.reset()\n",
    "                    \n",
    "    epoch_log[\"epoch\"] = epoch           \n",
    "    for name, metric in epoch_log.items():\n",
    "        history[name] = history.get(name, []) + [metric]\n",
    "\n",
    "    # 3，early-stopping -------------------------------------------------\n",
    "    arr_scores = history[monitor]\n",
    "    best_score_idx = np.argmax(arr_scores) if mode==\"max\" else np.argmin(arr_scores)\n",
    "    if best_score_idx==len(arr_scores)-1:\n",
    "        torch.save(net.state_dict(),ckpt_path)\n",
    "        print(\"<<<<<< reach best {0} : {1} >>>>>>\".format(monitor,\n",
    "             arr_scores[best_score_idx]),file=sys.stderr)\n",
    "    if len(arr_scores)-best_score_idx>patience:\n",
    "        print(\"<<<<<< {} without improvement in {} epoch, early stopping >>>>>>\".format(\n",
    "            monitor,patience),file=sys.stderr)\n",
    "        break \n",
    "    net.load_state_dict(torch.load(ckpt_path,weights_only=True))\n",
    "    \n",
    "dfhistory = pd.DataFrame(history)\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "jupytext": {
   "formats": "ipynb,md"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
